Learning Visual Context for Group Activity Recognition

Group activity recognition aims to recognize an overall in a multi-person scene. Previous methods strive reason on individual features. However, they under-explore the person-specific contextual information, which is significant and informative computer vision tasks. In this paper, we propose new reasoning paradigm incorporate global information. Specifically, two modules bridge gap between group visual context. The first Transformer based Context Encoding (TCE) module, enhances representation by encoding information features refining aggregated second Spatial-Temporal Bilinear Pooling (STBiP) module. It firstly further explores pairwise relationships for context encoded representation, then generates semantic representations via gated message passing constructed spatial-temporal graph. On their basis, design two-branch model that integrates designed into pipeline. Systematic experiments demonstrate each module's effectiveness either branch. Visualizations indicate cues can be globally TCE. Moreover, our method achieves state-of-the-art results widely used benchmarks using only RGB images as input 2D backbones.